Chemical  vapor  detection  with  a  multispectral 
thermal  imager 


Mark  L.  G.  Althouse,  member  spie 
U.S.  Army  Chemical  Research 
Development  and  Engineering  Center 
SMCCR-DDT 

Aberdeen  Proving  Ground,  Maryland 
21010 

Chein-I  Chang,  member  spie 
University  of  Maryland 
Department  of  Electrical  Engineering 
Baltimore,  Maryland  21228 


Abstract.  Detection  of  chemical  vapors  with  a  remote  sensor  is  necessary 
for  both  military  defense  and  civilian  pollution  control.  The  thermal  imager 
is  a  natural  instrument  from  which  to  build  a  chemical  sensor  since  most 
chemical  vapors  of  interest  are  spectrally  active  in  its  operating  wave¬ 
length  range.  A  system  has  been  designed  to  place  a  chemical  detection 
capability  as  an  adjunct  function  in  a  military  thermal  imager.  An  additional 
detector  array,  which  is  spectrally  filtered  at  the  focal  plane,  is  added  to 
the  imager.  Real-time  autonomous  detection  and  alarm  is  also  required. 
A  detection  system  model  by  Warren,  based  on  a  Gaussian  vapor  con¬ 
centration  distribution  is  the  basis  for  detection  algorithms.  Algorithms 
recursive  in  both  time  and  spectral  frequency  have  been  derived  using 
Kalman  filter  theory.  Adaptive  filtering  is  used  for  preprocessing  clutter 
rejection.  Various  components  of  the  detection  system  have  been  tested 
individually  and  an  integrated  system  is  now  being  fabricated. 

Subject  terms:  infrared  imaging  systems;  muitispectrai  thermai  imaging;  chemical 
detection;  adaptive  filters;  Kalman  filtering;  vapor  cloud  model. 
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1.  INTRODUCTION 

Detection  of  chemical  vapor  clouds  has  been  a  military  concern 
since  the  first  use  of  chemical  warfare.  Growing  concern  for 
environmental  pollution  has  additionally  driven  development  of 
remote  chemical  vapor  detectors.^  The  FLIR  (forward  looking 
infrared)  is  an  8-  to  12-p,m  thermal  imager  widely  used  in  the 
military  for  target  acquisition  and  night  vision.  Chemical  vapors 
are  spectrally  active  in  the  8-  to  12-p,m  region.  There  have  been 
several  military  programs  to  adapt  the  FLIR  for  chemical  de¬ 
tection.^"^  A  navy  FUDR  has  been  deployed  for  chemical  sensing."^ 
It  is  a  standard  common  module  FLIR  modified  with  bandpass 
spectral  filters.  Agent  detection  is  made  by  the  operator  based 
on  the  viewed  scene. 

Army  requirements  specify  an  autonomous  detection  system 
not  dependent  on  an  operator  for  an  alarm  decision.  To  meet 
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that  requirement,  the  Chemical  Research,  Development,  and 
Engineering  Center  (CRDEC),  as  part  of  its  detection  program, 
is  developing  automatic  image  enhancement  and  detection  al¬ 
gorithms.  A  large  body  of  work  exists  on  target  detection  and 
recognition  in  thermal  images.  But  the  targets  of  interest  in  this 
work  are  tanks,  trucks,  aircraft,  and  buildings,  all  objects  that 
are  generally  modeled  by  such  deterministic  functions  as  shape 
and  size.  Vapor  clouds  fit  no  fixed  size,  shape,  or  motion  cat¬ 
egories,  but  rather  are  modeled  statistically  based  on  their  con¬ 
centration.  The  detection  problem  becomes  one  of  deciding  on 
the  composite  hypothesis  of  the  image  containing  one  or  more 
distributions  of  various  chemical  vapors  versus  the  hypothesis 
of  the  image  containing  only  the  clutter  background.  Addition¬ 
ally,  any  detection  algorithm  must  operate  in  real  time,  that 
being  the  1/30-s  time  period  between  the  arrival  of  frames  of 
video.  To  meet  this  constraint,  adaptive  and  recursive  realiza¬ 
tions  are  pursued,  as  are  parallel  and  pipelined  implementations. 

Since  more  sensitive  remote  detection  methods  exist  for  a 
dedicated  chemical  sensor,  specifically  IR  interferometry^  and 
laser  spectroscopy,^  the  strengths  of  the  FLIR-based  approach 
are  the  pictorial  output  and  the  large  numbers  of  standard  FLIRs 
in  use  in  the  military.  Because  most  tactical  vehicles  and  aircraft 
are  fitted  with  an  FLIR,  the  space-  and  cost-economical  way  to 
provide  them  with  a  remote  chemical  vapor  sensing  capability 
would  be  through  inclusion  of  that  capability  in  the  FLIR.  As 
the  next  generation  of  FLIRs  is  developed,  an  adjunct  chemical 
vapor  detection  capability  could  be  incorporated  as  an  optional 
configuration. 

2.  SENSOR  CONCEPT 

Turning  an  FLIR  into  a  spectrometer  using  narrow-bandpass 
filters  is  a  fairly  simple  process.  Placing  a  filter  in  front  of  the 
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exit  aperture  of  the  FLIR,  much  as  one  does  with  a  filter  on  a 
camera,  is  the  initial  solution.  To  get  more  than  one  spectral 
band  requires  the  installation  of  a  filter  wheel  or  some  mechanical 
device  to  change  filters.  Of  course  when  the  expense  of  IR  filters 
several  centimeters  in  diameter  becomes  apparent,  a  quick  move 
is  made  to  an  internal  focal  point  in  the  FLIR.  This  was  the 
initial  approach*  at  CRDEC  and  it  performed  well  for  proof  of 
principle  and  initial  trials.  The  stepper-motor-controlled  filter 
wheel  could  reliably  switch  filters  every  second,  at  its  fastest. 
Thus  an  entire  set  of  four  bands  could  be  collected  in  about  10 
s.  Such  a  collection  rate  would  be  adequate  for  a  fixed  detector, 
i.e.,  nonmobile.  To  increase  the  data  collection  rate  the  next 
iteration  chemical  sensing  FLIR  has  a  continuously  spinning 
filter  wheel. This  wheel  has  three  different  passbands  and  is 
synchronized  to  the  video  frame  rate  of  the  FLIR,  thus  yielding 
a  full  set  of  spectral  data  every  three  frames  or  1/10  s.  A  0.1-s 
data  collection  time  would  be  adequate  in  slowly  moving  ve¬ 
hicles.  But  this  system  still  requires  a  fully  dedicated  FLIR  and 
would  experience  too  much  registration  error  for  use  in  aircraft. 

Dividing  the  8-  to  12-p,m  region  into  smaller  regions  in  an 
FLIR  can  be  accomplished  only  with  filters  or  a  dispersion  optic, 
both  of  which  have  transmission  losses.  Imaging  spectrometers 
based  on  other  principles  exist  but  do  not  have  the  spatial  res¬ 
olution,  compact  size,  or  high-data-acquisition  rate  of  the  FLIR.*° 
In  the  case  of  filters,  the  passband  may  be  as  narrow  as  0.5  p.m 
and  still  yield  an  80%  peak  transmission.  Filters  smaller  in  pass- 
band  quickly  degrade  in  peak  transmission.  Although  the  filter 
decreases  the  total  energy  incident  on  the  detector,  thus  lowering 
overall  sensitivity,  it  has  the  advantage  that  an  absorption  feature 
located  within  the  filter  passband  comprises  a  much  larger  per¬ 
centage  of  the  energy  incident  on  the  detector.  This  yields  a 
better  signal-to-noise  ratio  for  the  object  with  that  absorption 
feature.  Figure  1  illustrates  two  filter  passbands  and  the  spectrum 
of  SF6,  which  has  a  spectral  feature  in  the  10.6-p,m  filter  pass- 
band. 

There  are  two  difficulties  with  using  standard  bandpass  filters. 
To  divide  the  8-  to  12-p,m  band  fully  into  0.5-p,m-wide  segments 
would  require  eight  individual  filters.  These  need  to  be  me¬ 
chanically  rotated  into  the  field  of  view  sequentially  to  obtain 
spectral  data  over  the  full  range.  There  could  alternatively  be 
eight  to  ten  individually  filtered  detector  arrays,  each  with  a 
different  passband,  and  some  method  for  scanning  the  field  of 
view  over  each  detector  array.  Either  system  requires  an  optical 
redesign  of  the  FLIR.  A  second  difficulty  is  that  the  detector 
now  views  a  “hot’  ’  (system  operating  temperature)  filter  element 


Fig.  1.  Simulant  spectrum  with  passband  of  on-feature  and  off-fea- 
ture  spectral  filters. 


that  is  opaque  over  much  of  the  sensitivity  range  of  the  detector. 
Out-of-passband  emission  from  the  filter  represents  a  noise  source. 
This  may  not  be  a  problem  if  the  scene  background  is  signifi¬ 
cantly  warmer  than  the  filter,  but  if  a  colder  scene  is  viewed, 
the  warmer  filter  causes  a  considerable  loss  of  sensitivity.  An¬ 
other  concern  specific  to  tactical  military  FLIRs  is  the  require¬ 
ment  for  excellent  spatial  resolution  and  good  sensitivity  for 
target  acquisition  and  recognition.  The  image  quality  and  op¬ 
erational  availability  of  the  tactical  sensor  cannot  be  compro¬ 
mised  in  any  way  by  the  addition  of  further  mission  require¬ 
ments,  such  as  chemical  detection  or  hardware,  such  as  filters. 
To  address  all  of  the  necessary  elements  for  adjunct  chemical 
detection  in  a  harmonious  way  leaves  little  space  for  design 
alternatives  but  does  accommodate  a  solution. 

3.  HARDWARE  DESCRIPTION 

There  is  an  arrangement  that  would  permit  the  modified  FLIR 
to  deliver  both  its  standard  image  and  a  filtered  image  simul¬ 
taneously,  which  we  call  CSFLIR  for  chemical  sensing  FLIR. 
The  standard  image  would  not  be  degraded  in  any  way.  The 
filtered  image  would  be  designed  to  have  the  highest  sensitivity 
obtainable  for  the  given  detector  array  and  filter  bands.  Use  of 
both  the  filtered  and  unfiltered  images  allows  sufficient  spectral 
characterization  of  the  viewed  scene  to  detect  and  classify  chem¬ 
ical  vapor  clouds. 

Figure  2  illustrates  the  focal  plane  filtering  concept  for  a 
tactical  FLIR.  There  are  two  detector  arrays,  the  standard  tactical 
one  and  a  filtered  one  for  chemical  detection.  The  tactical  array 
is  a  standard  common  module  detector  array,  which  is  a  linear 
array  of  semiconductor  detector  elements,  generally  1  x  180, 
1  X  120,  or  1  X  60.  The  semiconductor  material  is  HgCdTe  for 
8-  to  12-p.m  detectors.  Individual  element  sizes  are  on  the  order 
of  4x10“^  by  6x10“^  mm.  The  signals  produced  on  these 
detector  elements,  as  a  scene  is  scanned  over  them,  are  elec¬ 
tronically  formed  into  a  video  image.  Dual  detector  implemen¬ 
tation  in  the  common  module  FLIR  would  require  a  redesign  of 
the  Dewar  assembly  to  accommodate  additional  electronic  con¬ 
nections. 

The  chemical  sensing  detector  array  would  be  identical  in 
construction  techniques  and  material  to  the  common  module 
(CM)  array.  It  could  have  elements  the  same  size  as  or  larger 
than  the  CM  array.  A  larger  detector  element  could  be  used 
because  it  is  more  sensitive  (less  noisy)  than  the  small  one.  The 
trade-off  is  a  loss  of  spatial  resolution  in  the  scene.  This  trade¬ 
off  has  been  found  not  to  be  a  great  concern  in  vapor  cloud 
sensing  as  vapor  clouds  tend  to  be  large  relative  to  the  scene 
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and  have  indistinct  edges.  High-spatial  resolution  is  necessary 
to  resolve  small  objects  or  sharp  edges  in  a  scene. 

A  focal  plane  filter  is  mounted  above  the  chemical  detector 
array  within  the  cooled  detector  housing.  It  consists  of  a  window 
material,  highly  transmissive  in  the  8-  to  12-p.m  band,  coated 
with  patches  of  narrow  bandpass  filter  coatings.  The  bandpass 
filters  are  made  by  vapor  depositing  multiple  layers  of  1/4-wave- 
length  thickness  dielectric  material  on  the  window  substrate.  A 
wide  variety  of  materials  are  used  and  the  deposition  process  is 
well  understood.  Adjacent  filter  patches  have  different  pass- 
bands.  The  number  of  distinct  passbands  required  depends  on 
the  complexity  and  number  of  chemical  vapors  being  detected. 
A  passband  should  be  associated  with  each  strong  absorption 
feature  and  at  least  one  band  where  there  are  no  features  for 
baseline  or  background  reference.  The  filter  patches  would  be 
arranged  in  order  from  1  to  n,  repeating  the  order  till  the  whole 
detector  array  is  covered,  as  shown  in  Fig.  3.  The  physical  depth 
of  some  of  the  passband  coatings  is  on  the  order  of  the  CM  array 
element  dimensions.  As  such  it  may  not  be  possible  to  put  a 
filter  patch  over  each  element,  but  the  elements  would  be  grouped; 
say  four  elements  under  a  patch,  one  element  lost  in  the  transition 
region  from  one  patch  to  the  next,  then  four  more,  and  so  on, 
as  in  Fig.  4.  Any  number  of  detector  elements  could  be  placed 
under  the  filter  such  that  the  filter  coating  edge  should  not  in¬ 
terfere  with  the  detector  element  acceptance  angle.  The  smallest 
groupings  and  hence  the  largest  number  of  filter  strips  is  desirable 
to  get  the  most  complete  multispectral  coverage  on  the  image. 
A  custom  array  with  larger  detectors  could  be  matched  to  the 
design  constraints  of  the  filter  patches,  as  in  Fig.  3.  The  filter 
patch  should  be  at  least  three  times  as  wide  as  it  is  deep  for 
structural  stability.  A  filter  patch  edge  with  no  slope  is  preferable 
as  the  transition  will  then  occupy  less  space  on  the  image  plane. 
Each  filter  element  cannot  have  an  edge  of  the  filter  patch  within 
its  acceptance  angle  without  experiencing  serious  focusing  and 
spectral  errors.  The  filters  should  be  mounted  far  enough  from 
the  detector  to  avoid  large  noise  contributions  due  to  forward 
scattering  from  small  defects,  which  generally  exist  in  the  filter 
patches.  This  scattering  is  called  the  Stierwalt  effect. 

Since  the  filters  are  fabricated  from  1/4- wavelength  thickness 
layers  sequentially  applied,  any  foreign  material  that  adheres  to 
the  filter  during  fabrication  will  result  in  a  feature  as  shown  in 
Fig.  5.  The  bump  created  by  the  defect  acts  as  a  lens  to  focus 
stray  radiation  onto  the  detector. 

Commercial  infrared  filters  generally  contain  some  of  these 
defects.  Because  the  filters  are  not  generally  mounted  close  to 
the  detector,  the  defects  have  no  effect.  Through  careful  pro- 


a.  WINDOW  MATERIAL 


b.  CUSTOM  DETECTOR  ELEMENT 

c.  BANDPASS  FILTER  PATCH 


Fig.  3.  Top  view  of  a  focal  plane  filter  covering  a  custom  detector 
array;  one  filter  patch  per  detector  element. 


a.  WINDOW  MATERIAL 
C.  BANDPASS  FILTER  PATCH 

d.  COMMON  MODULE  DETECTOR  ELEMENT 

e.  DETECTOR  ASSEMBLY  SUBSTRATE 

f.  ACCEPTANCE  ANGLE 

Fig.  4.  Side  view  of  a  focal  plane  filter  covering  a  common  module 
detector  array;  four  detector  elements  per  filter  patch. 


Fig.  5.  Effect  on  filter  coating  layers  due  to  a  small  particulate  im¬ 
purity. 


Fig.  6.  Off  passband  transmittance  from  a  spectral  filter  as  a  function 
of  the  half-cone  angle  of  the  detector. 


cessing,  we  can  greatly  reduce  the  number  of  these  defects  and 
thus  eliminate  the  consequent  noise  of  the  Stierwalt  effect.  Fig¬ 
ure  6  shows  some  data  from  Stierwalt  that  illustrate  the  effect.** 
The  curve  shows  transmittance  of  an  out-of-passband  frequency 
for  various  half-cone  acceptance  angles  of  the  detector.  As  we 
move  a  filter  with  fixed  aperture  closer  to  the  detector,  the  half¬ 
cone  angle  increases,  to  a  limit  of  90  deg.  Figure  7  illustrates 
the  change  in  half-cone  angle  with  the  change  in  offset  distance 
of  the  filter  from  the  detector  for  a  fixed  aperture.  Defects  in 
the  filter  coating  not  only  scatter  the  object  beam  away  from  the 
detector,  thus  reducing  desired  signal  strength,  but  scatter  and 
focus  stray  radiation  from  outside  the  normal  optical  path  and 
out-of-band  radiation  onto  the  detector,  thus  increasing  the  noise 
level.  The  larger  the  acceptance  angle  of  the  detector  the  more 
pronounced  the  effect. 

Some  recent  measurements  with  high-quality  filters  indicated 
no  Stierwalt  effect  up  to  a  half-cone  angle  of  20  deg.  Initial 
designs  for  a  focal-plane-filtered  FLIR  include  half-cone  angles 
greater  than  20  deg.  As  a  result,  great  care  is  being  taken  to 
ensure  a  clean  atmosphere  and  pure  materials  for  the  ongoing 
initial  fabrication  of  these  filters. 
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OFFSET  DISTANCE  (RELATIVE  UNITS) 


Fig.  7.  Change  in  haH-cone  angie  with  varying  offset  distances  for 
a  fixed  aperture  50%  iarger  than  the  detector  size.  Units  are  nor¬ 
malized  to  the  dimension  of  the  detector. 

The  image  resulting  from  the  described  chemical  sensing 
detector  would  appear  as  in  Fig.  8.  Horizontal  stripes  of  the 
image  would  be  filtered  by  different  passbands  in  a  repeating 
order  vertically  down  the  image.  A  building  or  solid  feature  that 
has  a  flat  emission  spectrum  would  show  only  minor  changes 
in  appearance  from  stripe  to  stripe  due  to  slightly  unequal  band¬ 
pass  functions.  On  the  other  hand,  the  vapor  cloud,  which  has 
distinct  spectral  properties,  is  strongly  visible  through  filter  1, 
weakly  visible  through  filter  3,  and  not  visible  through  filters  2 
and  4.  To  an  operator  this  would  be  a  very  poor  quality  image, 
but  to  an  image  processor,  its  analysis  poses  no  problem.  Al¬ 
though  the  system  does  not  produce  a  complete  set  of  spectral 
data  for  each  pixel,  it  does  provide  sufficient  spectral  analysis 
of  the  scene.  RS-170  standard  video  is  composed  of  480  hori¬ 
zontal  lines.  Using  the  assumption  of  four  filtered  elements,  the 
next  element  lost  to  filter  patch  transition,  four  more  filtered 
elements,  and  so  on,  we  would  get  96  stripes  in  the  image.  The 
next  generation  of  tactical  FLIRs  will  have  480-line-image  data. 
Given  the  180-line  CM  display,  we  would  get  36  stripes.  With 
four  distinct  passbands,  that  would  yield  six  complete  filter  se¬ 
quences.  Based  on  tests  with  vapor  clouds  generated  in  a  realistic 
manner,  the  cloud  quickly  fills  a  significant  portion  of  the  image 
if  it  is  close  enough  to  be  detected.  The  FLIRs  used  in  the  study 
had  fields  of  view  (FOV)  on  the  order  of  20x30  deg.  Many 
tactical  FLIRs  have  a  much  smaller  FOV  and  thus  will  have  the 
cloud  fill  a  large  portion  of  the  FOV. 

Another  advantage  of  this  system  over  the  use  of  discrete 
filters  in  a  filter  wheel  or  a  continuously  variable  filter  is  its 


Fig.  8.  Striped  appearance  of  the  filtered  image. 


speed  of  data  acquisition.  Most  image-processing  techniques 
applicable  to  the  cloud  detection  problem  require  good  image 
stability  and  registration  over  the  sequence  of  images  used  for 
analysis.  Tactical  FLIRs  tend  to  be  mounted  on  moving  vehicles, 
resulting  in  motion  in  the  seene  between  frames.  Correcting  the 
sensor  motion  is  possible  but  computationally  costly.  In  the  new 
design  both  filtered  and  unfiltered  images  are  collected  simul¬ 
taneously  at  the  system  frame  rate,  generally  30  Hz.  The  actual 
filtered  pixel  to  corresponding  unfiltered  pixel  collection  time 
difference  will  be  on  the  order  of  10  s  due  to  the  physical 
separation  of  the  detector  arrays  on  the  focal  plane.  For  systems 
with  a  stepped  filter  wheel  or  continuously  variable  filter  that 
same  pixel  collection  time  difference  is  on  the  order  of  seconds. 
In  the  case  of  the  continuously  spinning  filter  wheel,  even  this 
most  ideal  mechanically  scanned  system  will  have  a  pixel  col¬ 
lection  time  difference  of  at  least  1/30  s.  A  10”^  s  time  sepa¬ 
ration  will  keep  any  scene  blurring  at  subpixel  resolution  even 
if  the  FLIR  is  mounted  in  a  high-speed  aircraft.  Thus  this  system 
is  compatible  with  present  detection  algorithms.  The  only  im¬ 
provement  in  the  amount  and  usefulness  of  information  collected 
would  come  from  the  addition  of  more  detector  arrays,  each 
filtered  at  a  single  color,  providing  simultaneous  multispectral 
data  for  each  pixel. 

The  block  diagram  in  Fig.  9  shows  the  major  system  com¬ 
ponents.  The  FLIR  optics  and  tactical  detector  are  as  designed 
for  a  standard  CM  FLIR  application.  The  CSFLIR  detector  and 
tactical  detector  are  integrated  into  a  detector/cooler  assembly 
that  includes  a  cryogenic  pump.  A  data  line  from  each  detector 
carries  the  signal  to  video  electronics  modules  that  transfer  it  to 
RS-170  format.  Both  RS-170  lines  feed  a  digital  image  proces¬ 
sor.  The  image  processor  carries  out  the  image  enhancement 
routines  called  in  the  detection  algorithm.  Prior  to  the  actual 
decision  on  the  presence  of  a  cloud,  the  detection  algorithm 
controls  the  processing  and  extraction  of  relevant  information 
contained  in  the  available  data. 

Status  of  the  chemical  vapor  detection  operation  is  presented 
to  the  system  operator  via  an  audio  or  visual  means.  The  operator 
could  have  manual  selection  of  an  appropriate  display  mode  in 
his  image  display  screen.  It  may  be  a  colored  area,  shaded  area, 
or  perimeter  overlay  of  the  detected  vapor  cloud. 


Fig.  9.  Major  system  components. 
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4.  SYSTEM  MODEL 

For  a  standard  target  detection  problem  in  the  FLIR  world,  the 
target  is  a  hard  object  that  has  a  relatively  fixed  dimension  and 
profile  and  produces  a  sharp-edged  feature  in  the  FLIR  image. 
Detection  algorithms  of  many  types  have  been  devised  to  sep¬ 
arate  features  with  those  characteristics  from  the  scene  back¬ 
ground  or  clutter.  Clutter  itself  is  modeled  as  a  first-order 
Markov  process  and  statistical  methods  have  been  devised  for 
its  suppression.  Rauch  et  al.  treated  normal  atmospheric  clouds 
as  clutter  and  investigated  methods  to  suppress  them.  Minor 
and  Sklansky  presented  a  method  for  detecting  blobs,  a  class 
into  which  clouds  can  fall,  in  IR  images  using  edge  detection 
and  other  techniques  designed  for  deterministically  represented 
features.**  We  have  found  no  other  work  in  which  clouds  have 
been  treated  as  a  target  for  a  detection  algorithm.  Since  vapor 
clouds  have  certain  properties  that  differ  from  clutter,  even  cloud 
clutter,  these  properties  can  be  the  basis  of  a  discrimination 
technique. 

In  order  to  design  a  chemical  agent  detector  for  the  multi- 
spectral  FLIR,  a  sufficiently  accurate  system  model  is  required. 
It  must  encompass  the  background,  signal  from  the  vapor  cloud, 
and  the  sensor.  The  model  described  in  this  section  is  due  to 
Warren. We  summarize  briefly  his  derivation. 

The  extinction  coefficient  at  wavelength  k 


ex(x,z)  =  eax-t-pxC(x,z)  , 


(1) 


where  x  =  (jc,y)  represents  a  two-dimensional  spatial  image,  SaX 
is  an  ambient  atmospheric  component,  px  is  the  absorbtivity  of 
the  vapor,  and  C(x,z)  is  the  vapor  concentration  at  location  x 
in  the  plane  at  z,  normal  to  the  instrument  line  of  sight.  A 
Gaussian  model  for  the  vapor  concentration  is  used: 


C(x,z)  =  Co  exp 


2al 


where  x,  is  the  pixel  location  on  the  detector  plane,  0  is  the  field 
of  view  direction  of  the  pixel,  /  is  the  effective  focal  length, 
and  Pn  is  the  additive  detector  noise  uncorrelated  with  the  signal. 
Signal  power  Pd  is  considered  a  random  function  since  both 
detector  noise  and  the  background  temperature  rg(0)  are  random 
functions.  Thus  we  can  characterize  Pd  by  its  first-  and  second- 
order  moments.  First  we  generalize  Eq.  (4)  for  a  time  (index  k) 
series  of  multispectral  (index  j)  images, 

Pdp<-i,tk)=  |p,,(0,rt)/f(Xi//-0)d20  +  />„/Xi)  ,  (5) 

where  the  entrance  aperture  power  is 


P,/0,4)=A 


Bx(Tc)  +  TA(k)  exp[-pxCL(0,t;t)] 


x{Bx[Tb(0,4)]-Sx(7’c)} 


dk  . 


Additional  parameters  are 


(6) 


A  =  system  optical  constant  , 

B\{T)  =  Planck  function  at  temperature  T, 

Fj{\)  =  bandpass  function  of  the  optical  filter  centered  at 
wavelength  ky, 

Tc  =  temperature  of  the  vapor  cloud,  and 

TA(k)  =  transmittance  of  the  atmosphere  at  wavelength  k. 

The  first  and  second  moments  of  the  signal  power  are 

E[Pdjixi,tk)]^Pdj{T^i,tk)=  {  E[Ps/e,tkmxi/f-e)  (?) 

and 


with  Co  the  peak  concentration  and  zo  the  cloud  center.  Since 
an  FLIR,  as  a  passive  instrument,  collects  a  signal  integrated 
over  the  total  path  length,  the  path-integrated  concentration  CL(x,z) 
is 

CL(x,z)=(  C(x,z')dz' 

-'0 


Apy^.,(x,-  ,4  ,x,'  ,tk')=E{[Pdj(XiA) = Pdj(Xi,tk)]lPdf(Xi',tk') 

-PdfCxi'Jk')]} 

=  |[R(x,//-0)A^,(0,4;0',4') 

xR(x,'//-0)]  d^0  d^0' 


(2) 


(8) 


(9) 


=  VSirCoac  exp 


(x^+y^) 

2a? 


a< 


-e 


Z-Zq 

(Tc 


with  <2(x)  defined  by 
Q(x)  =  f  exp(  — 1^/2)  dr  . 


+  A„j,-.(xi,tk;xr,tk')  ,  (10) 

where  l^k^M,  and  l^y'^L. 

Background  or  clutter  temperature  js  assumed  to  be  wide 
sense  stationary  such  that  £[7^(0, r*)]  =  Tb,  a  constant.  Although 
background  scenes  do  not  generally  exhibit  stationarity,  War¬ 
ren  found  that  his  algorithm  performed  well  with  field  data. 
Using  these  two  moments,  the  signal  model  is  a  multivariate 
Guassian  density  with  signal  power  Pd  expressed  as  an 
L-dimensional  random  vector  Pd  =  (Pd„  •  •  • ,  PdJ  with  probability 
density  function  given  by 


The  detected  signal  power  Pd  is  a  convolution  of  Ps,  the 
power  at  the  entrance  aperture  of  the  sensor,  with  R,  the  system 
point  spread  function. 


Pd(Xi 


->=/ 


P,(0)R(x,//-0)  d20-fP„(x,) 


(4) 


X  expj  --I 


d 


(11) 
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Using  the  probability  density  function,  Eq.  (11),  yields  the  fol¬ 
lowing  likelihood  ratio  test: 


i(Pd)  = 


Ht 

^pXPdlHr)  ^  PjHo) 
‘3‘pXVd\Ho)  Ho  P{Ht) 


-1  > 


where  Ho  is  the  hypothesis  that  no  vapor  exists  and  Ht  is  the 
hypothesis  that  a  vapor  cloud  target  is  present. 

The  above  likelihood  ratio  can  be  further  reduced  to  the  sum¬ 
mation  over  the  time  and  spectral  dimensions  of  a  function  Rkji'H-i), 
where  Rkj(xi)  is  arrived  at  by  taking  the  product  of  the  Fourier 
transform  of  each  input  image  at  each  time  increment  and  spectral 
frequency  with  the  inverse  of  the  autocovariance  at  each  spectral 
frequency,  the  target  filter  function,  and  the  sensor  modulation 
transfer  function  (MTF).  Here  /?/y(x,)  is  the  result  of  the  inverse 
Fourier  transform  of  the  above  product.  Computing  the  inverse 
of  the  autocovariance  turned  out  to  be  the  most  time-consuming 
process.  When  implemented  in  FORTRAN  on  a  Micro  VAX 
computer,  a  64-frame  sequence  of  two  spectral  bands  required 
more  than  I/2  h  to  process.  The  resulting  images  had  a  very 
significant  increase  in  signal-to-noise  ratio  and  targets  became 
easy  to  detect  visually.*^ 

5.  PREPROCESSING  ALGORITHMS:  BACKGROUND 
CLUTTER  SUPPRESSION 

FLIR  sensors  image  the  thermal  radiation  emitted  by  a  target. 
The  target  presence  is  identified  by  a  difference  in  temperature 
between  the  target  and  the  immediate  surroundings,  i.e.,  back¬ 
ground  clutter  and  noise.  As  a  result,  the  loss  of  the  contrast 
between  the  target  and  the  background  can  seriously  degrade 
the  detectability  of  the  target.  In  order  to  alleviate  this  problem, 
preprocessing  is  generally  required  prior  to  thresholding  detec¬ 
tion.  Two  preprocessing  techniques  are  commonly  used: 
(1)  adaptive  image  enhancement^*  and  (2)  background  suppres¬ 
sion.  Of  particular  interest  in  multispectral  images  is 

the  latter  approach,  which  designs  computationally  efficient 
adaptive  filters  of  different  types  to  remove  or  suppress  highly 
structured  background  clutter.  Since  an  adaptive  filter  has  the 
capability  of  adapting  unknown  statistics  and  yields  desired  re¬ 
sults  over  a  wide  range  of  environments,  the  study  of  adaptive 
filters  for  a  variety  of  applications  has  received  considerable 
interest  over  the  past  years.^"* 

A  widely  used  technique  for  adaptive  multispectral  filtering 
is  to  use  the  minimum  mean-square-error  (MMSE)  criterion  to 
achieve  background  clutter  suppression,  thus  improving  detect¬ 
ability  of  targets  in  FLIR  images. **’*’’^^  The  adaptive  filter  to 
be  designed  consists  of  a  set  of  scalar  filter  coefficients  expressed 
by  a  weighting  vector  w.  The  output  of  the  filter,  y  is  the  weighted 
sum  of  the  input  signal  vector  x  given  by 

y=w^x  ,  (12) 

where  T  is  the  transpose  and  w  is  a  weighting  vector.  An  optimal 
adaptive  filter  is  one  minimizing  the  mean-square  error  between 
the  desired  signal  and  the  output  y. 

Two  approaches  to  finding  an  optimal  weighting  vector  for 
a  desired  MMSE  adaptive  multispectral  filter  are  of  interest. 

5.1.  Unconstrained  MMSE  adaptive  filtering 

FLIR  image  background  suppression  can  be  cast  as  an  uncon¬ 
strained  MMSE  adaptive  filter  problem. The  resulting 


optimal  filter  is  generally  referred  to  as  a  Wiener-Hopf  filter. 
To  be  more  specific,  let  e  be  the  difference  between  the  desired 
signals  s  and  y  expressed  by 

e  =  s-y  .  (13) 

An  optimal  filter  minimizes  the  mean  square  of  Eq.  (13),  i.e., 
£'(e^)  =  £[(s  — y)^(s  — y)],  where  E  is  the  expectation  taken  with 
respect  to  the  joint  random  vector  (s,y).  The  resulting  optimal 
weighting  vector  Wopt  is  a  solution  to  the  well-known  Weiner- 
Hopf  equation: 

Wopt~RxD  ,  (14) 

where  R*  is  the  inverse  of  Mx  =  £[xx^],  the  covariance  matrix 
of  X,  and  D  =  £[xs]  is  the  cross  covariance  matrix  of  x  and  s. 

We  use  a  window  (or  search  box)  to  implement  adaptive 
filtering  to  suppress  the  background  clutter  of  FLIR  images.  The 
window  is  moved  and  centered  on  all  pixels  in  the  image  in  turn 
to  calculate  all  local  covariance  matrices.  The  window  consid¬ 
ered  here  is  an  adaptive  filter  specified  by  a  weighting  vector 
that  puts  weights  on  all  pixels  falling  in  the  window.  The  details 
of  implementation  can  be  found  in  Refs.  16,  17,  and  23. 

5.2.  Linearly  constrained  MMSE  adaptive  filtering 

An  alternative  approach  to  the  unconstrained  MMSE  adaptive 
filtering  is  a  constrained  MMSE  adaptive  filtering, which  is 
applied  to  the  case  where  prior  knowledge  of  the  desired  signal 
s  is  not  known.  Due  to  the  lack  of  s,  a  linear  constraint  vector 
c  must  be  imposed  on  the  filter  to  constrain  the  filter  output.  Let 
the  linear  constraint  be  given  by 

Jyv  =  g  .  (15) 

The  resulting  constrained  MMSE  adapative  filter  is  characterized 
by  finding  an  optimal  weighting  vector  Wopt,  a  solution  to  the 
following  equation: 

imn[w^MxW]  subject  to  c^w  =  g  ,  (16) 

where  Wopt  can  be  solved  and  given  by 


Using  the  formulation  (16),  the  adaptive  filtering  for  FLIR 
background  suppression  can  be  accomplished  by  choosing  an 
appropriate  linear  constraint  vector  c  and  a  gain  g.  For  instance, 
if  we  are  interested  in  applying  a  3  x  3  square  array  window  to 
an  image  while  maintaining  the  target  signal  unchanged,  the 
gain  g  can  be  chosen  to  be  unity  and  the  constraint  vector  c 
chosen  as  a  nine-dimensional  vector  with  one  in  the  central 
component  and  zero  in  all  other  components.  As  a  result,  the 
pixel  falling  in  the  center  of  the  window  is  retained  unchanged. 
This  formulation  yields  a  constrained  MMSE  problem.  The  re¬ 
sulting  optimal  filter  is  generally  referred  to  as  a  minimum  var¬ 
iance  distortionless  response  (MVDR)  adaptive  beamformer. 

5.3.  Systolic  array  algorithms 

A  major  difficulty  encountered  in  the  approaches  described  in 
Sec.  5.1  and  5.2  is  the  computation  of  the  optimal  weighting 
vector  resulting  from  adaptive  filters,  which  requires  inverting 
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a  sample  covariance  matrix  M*  and  intensive  large-scale  matrix/ 
vector  multiplications.  It  is  very  costly  if  the  filter  is  implemented 
with  direct  matrix  inversion.  Although  Wang^^  developed  a  re¬ 
cursive  formula  for  generating  the  optimal  weight  vector  to  re¬ 
duce  computational  load,  full  advantage  of  using  matrix  struc¬ 
tures  is  not  exploited. 

Recently,  the  introduction  of  systolic  array  algorithms  by 
Kung  and  Leiserson^®  revolutionized  hardware  design  for  car¬ 
rying  out  matrix  computations.  With  the  help  of  systolic  array 
algorithms,  matrix  computations  can  be  performed  in  parallel 
and  implemented  in  real-time  processing.  Basically,  systolic  ar¬ 
ray  algorithms  are  orthogonal  triangularization  processes  using 
QR-decomposition  or  Cholesky  factorization,  which  allow  us  to 
triangularize  the  sample  covariance  matrix  so  that  the  resulting 
triangularized  matrices  can  be  inverted  very  efficiently  by  for¬ 
ward  and  backward  substitutions.  Most  importantly,  the  systolic 
array  algorithms  are  numerically  stable,  robust  to  finite  arith¬ 
metic  precision,  and  can  be  designed  to  be  parallel  and  pipelined, 
thus  greatly  improving  the  implementation  of  the  optimal  filter 
in  real-time  applications.  The  details  of  studying  such  a  systolic 
array  approach  are  reported  by  Ref.  27. 


6.  REAL-TIME  DETECTION  ALGORITHMS 

After  preprocessing  FLIR  images,  a  thresholding  technique  is 
applied  to  detect  the  target  signal.  Many  detection  algorithms 
have  been  developed  for  this  purpose.  Among  them  is  the  de¬ 
tection  algorithm  suggested  by  Warren, particularly  note¬ 
worthy  because  it  is  recursive  in  time  and  has  been  shown  to  be 
effective. 

Rather  than  following  Warren’s  algorithm,  which  is  based 
on  a  first-order  autoregressive  (AR)  time  series,  Kalman  filter 
theory  is  used  to  develop  a  similar  recursive  detection  algorithm 
that  covers  Warren’s  algorithm  as  a  special  case  and  extends  it 
to  include  recursion  in  spectral  frequency.  The  idea  of  using  a 
Kalman  filter  is  natural  since  it  can  be  recursively  implemented 
in  real-time  processing.  Unlike  Warren’s  work,  this  approach 
uses  a  state  equation  to  model  background  clutter  of  different 
types.  The  background  clutter  in  Warren’s  work  was  assumed 
to  be  a  first-order  autoregressive  model  that  corresponds  to  its 
counterpart,  a  state  equation  in  Kalman  filtering.  Using  the  Mar¬ 
kov  property  induced  by  the  AR  model,  Warren  derived  a  re¬ 
cursive  formula  for  detectors  described  by  a  sequential  likelihood 
ratio  test  statistic.  The  essence  of  Kalman  filter  theory  is  to 
introduce  a  new  process,  the  innovations  process  suggested  by 
Kailath.^®  Instead  of  directly  dealing  with  an  observation  pro¬ 
cess,  an  innovations  process  is  generated  for  updating  new  in¬ 
formation  as  time  goes  along.  In  other  words,  given  an  obser¬ 
vation  process  it  is  not  necessary  to  store  all  information  available 
up  to  the  processing  time  because  some  information  will  be 
useless  and  some  will  be  repeatedly  stored,  which  wastes  stor¬ 
age.  A  more  efficient  way  to  manage  information  is  to  store  all 
necessary  information  only  once  and  dump  unnecessary  or  un¬ 
wanted  information.  The  innovations  process  is  developed  based 
on  this  need.  In  general,  an  observation  process  can  be  decom¬ 
posed  into  two  processes,  a  predicted  process  and  an  unpredicted 
process.  The  predicted  process  contains  all  previous  information 
required  for  processing,  and  the  unpredicted  process  presents 
new  information  available  at  the  processing  time  but  not  con¬ 
tained  in  the  predicted  process.  Such  an  unpredicted  process 
resulting  from  the  observation  process  is  generally  referred  to 
as  an  innovations  process. 


In  most  practical  cases,  the  background  at  time  frame 
k  can  be  characterized  by  a  first-order  AR  model  (or  state  equa¬ 
tion)  given  by 

Bk(u)  =  y(x)Bk-ii'K)  +  nk(x)  ,  (18) 

and  the  observation  process  specified  by  a  spatial  image,  sampled 
at  time  k  is  given  by  a  random  process  /*(x).  A  standard  detection 
problem  can  be  described  by  a  binary  hypothesis-testing  problem 
given  as  follows.  At  time  frame  k, 

Ho:  Ik(x)=Bk(x)+Nk(x)  (19) 

versus 

Hi:  Ik(x)  =  Tkix)  +  Bk(x)+Nkix)  ,  (20) 

where  r*(x)  is  the  target  of  interest  and  both  Nk(x)  in  Eqs.  (19) 
and  (20)  and  n*(x)  in  Eq.  (18)  are  noises. 

Instead  of  using  4(x)  and  B*(x),  we  can  use  two  innovations 
processes  a*(x)  and  e*(x)  to  replace  4(x)  and  Bk(x)  in  Eqs.  (19) 
and  (20),  respectively,  where  at(x)  and  et(x)  are  given  by 

ak(x)=Ik{x)-lk\k-i('s.)  (21) 

^k(x)=Bkix)-Bk\k-i(^)  ,  (22) 

and  both  4|*-i(x)  and  Byt|(t-i(x)  are  the  estimates  of  4(x)  and 
Bk(x)  at  time  k  and  are  obtained  based  on  previous  information 
up  to  time  frame  A:  —  1.  As  a  consequence,  a  detection  problem 
using  a  Kalman  filter  with  the  state  model,  Eq.  (18),  can  be 
derived  and  given  by 

Ho:  akix)=Sk\o(x)  +  ekix)+Nk(x)  (23) 

versus 

Hi:  akix)  =  Sk\iix)  +  €k(x)+Nkix)  ,  (24) 

where  iqo(x)  =  0  for  Ho,  no  target  present,  and  5*|i(x)  =  4t(x) 
for  Hi ,  target  present. 

As  we  can  see  from  the  above,  the  innovations  process  at(x) 
is  a  process  containing  the  new  information  available  in  the 
process  4(x)  at  time  k  but  not  in  4|(t-i(x),  where  4|jt-i(x)  is 
the  predicted  process  obtained  from  the  past  information  up  to 
time  k— 1.  Similarly,  €*(x)  is  the  innovations  process  repre¬ 
senting  the  new  information  provided  by  the  state  model  Bk(x) 
at  time  k,  but  not  available  in  the  past.  It  is  obtained  by  sub¬ 
tracting  the  predicted  process  B*|*-i(x)  from  Bk(x)  so  that  the 
predicted  information  from  fi,(x)  up  to  time  /  =  0,  ...,  —  1  can 
be  removed.  One  of  the  most  important  features  these  two  in¬ 
novations  processes  possess  is  that  they  are  white,  i.e.,  inde¬ 
pendent  processes.  This  is  not  true  for  the  detection  model  of 
Eqs.  (18)  through  (20)  considered  by  Warren.  The  property  of 
independency  makes  detection  problems  extremely  easy  to  deal 
with.  The  observation  of  such  independency  is  easily  justified. 
Since  an  innovations  process  contains  unpredicted  new  infor¬ 
mation  obtained  at  different  time  frames,  the  information  ob¬ 
tained  at  a  certain  time  frame  must  be  independent  of  other  time 
frames  due  to  the  nature  of  unpredictability.  The  relationship 
between  the  two  innovations  processes  can  be  demonstrated  by 
the  following  equation: 
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akix)  =  Bkix)+Nkix)-Bk\k-i{^)  (25) 

=  ek{x)+Nk(.x)  .  (26) 

Another  great  advantage  of  using  a  Kalman  filter  over  War¬ 
ren’s  approach  is  that  the  derived  formula  can  be  extended  to 
more  general  models  for  background  clutter  other  than  AR  models. 
The  details  of  development  of  a  Kalman-filter-based  detection 
algorithm  can  be  found  in  the  report  by  Chang. 

Although  algorithms  to  date  are  designed  for  a  three-dimen¬ 
sional  data  set  (pixel,  time,  spectral  frequency),  and  the  CSFLIR 
data  will  be  more  of  the  form  [spectral  frequency  (pixel),  time], 
separation  of  the  spectral  frequency  function  into  its  linear  com¬ 
ponents  is  really  a  bookkeeping  process.  Comparison  of  adjacent 
components  (really  the  horizontal  stripes  in  the  image)  with  the 
associated  unfiltered  pixels  provides  a  sufficiently  complete  set 
of  data  due  to  the  correlation  between  neighboring  pixels. 

7.  CONCLUSIONS 

Remotely  detecting  chemical  vapors  with  a  thermal  imager  has 
proven  to  be  a  viable  technique.  Operational  constraints  drive 
the  degree  of  modification  necessary  to  convert  a  standard  FLIR 
into  a  chemical  sensor.  The  developmental  FLIR  design  pre¬ 
sented  here  would  yield  a  compact  multipurpose  electro-optical 
system  for  obtaining  multispectral  imagery  from  a  moving  plat¬ 
form.  The  multispectral  data  would  be  stationary  and  complete 
enough  for  the  planned  detection  algorithms.  Hardware  technical 
issues,  such  as  scattering  from  defects  in  the  filters,  exist  but 
have  engineering  solutions  and  require  no  new  science. 

Detection  algorithm  work  accomplished  to  date  suggests  a 
real-time  solution  to  autonomous  operation.  Adaptive  filtering 
for  clutter  rejection  and  recursive  realizations  of  thresholding 
detectors  must  be  further  refined  with  realistic  data  and  then 
implemented  on  systolic  array  and  parallel  hardware.  Addition¬ 
ally,  neural  networks  may  perform  well  for  circumstances  where 
little  or  no  prior  knowledge  of  the  target  exists. 

The  potential  applications  of  this  technique  are  not  limited  to 
chemical  vapor  detection  or  remote  sensing.  Any  scene  that 
contains  hard  to  resolve  or  differentiate  objects  that  also  have 
spectral  features  distinct  from  the  background  can  be  analyzed 
by  an  imager  such  as  the  one  described  here.  Some  examples 
are  medical  imaging  or  industrial  process  control.  It  is  also 
possible  to  extend  the  spectral  range  beyond  the  8-  to  12-(xm 
region  used  in  this  description.  The  constraints  come  from  the 
limited  spectral  bands  of  optical  components,  but  the  use  of  all 
or  nearly  all  reflective  optics  is  helpful.  More  than  the  two 
detectors  shown  in  this  description  may  also  be  used  to  ade¬ 
quately  cover  the  spectral  range  needed. 
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